NSF - CAREER : The Listening Machine Annual Report 2004

This year, we expanded our investigation of sound analysis to look at a range of different sound 'scenes', including overlapping conversations, ambient everyday sound, and music. In each case, the goal is to abstract useful information similar to that which a human listener would perceive, and in particular to deal successfully with the issues raised by multiple, overlapping sound sources. Our most focused effort was the continued development of the novel model for sound sources we proposed last year, based on treating each spectral instant as a simple deformation of its immediate predecessor (or, in general, its neighbors). This model decomposes smoothly-varying segments of sound into a single spectral profile, and a set of locally-smooth transformation functions, describing how the spectral detail is derived from its predecessors. This year, we extended this model to a two-layer version with separate transformations applied to fine spectral structure (e.g. harmonics, to account for changes in pitch) and broader spectral structure (e.g. the formants of voice, which in general will move independently of the harmonics). The key to this model is the way that the parame